THE  CORRELATION  OF  HUMAN  CAPITAL  ON  COSTS 
OF  AIR  FORCE  ACQUISITION  PROGRAMS 

THESIS 


Stephen  E.  Gray,  Captain,  USAF 
AFIT/GCA/ENV/09-M05 


DEPARTMENT  OF  THE  AIR  FORCE 
AIR  UNIVERSITY 

AIR  FORCE  INSTITUTE  OF  TECHNOLOGY 


Wright-Patterson  Air  Force  Base,  Ohio 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 


The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect  the  official 
policy  or  position  of  the  United  States  Air  Force,  Department  of  Defense,  or  the  United 
States  Government. 


AFIT/GCA/ENV/09-M05 


THE  CORRELATION  OF  HUMAN  CAPITAL  ON  COSTS  OF  AIR  FORCE 

ACQUISITION  PROGRAMS 

THESIS 

Presented  to  the  Faculty 

Department  of  Systems  and  Engineering  Management 
Graduate  School  of  Engineering  and  Management 
Air  Force  Institute  of  Technology 
Air  University 

Air  Education  and  Training  Command 
In  Partial  Fulfillment  of  the  Requirements  for  the 
Degree  of  Master  of  Science  in  Cost  Analysis 


Stephen  E.  Gray,  BBA,  MSM 
Captain,  USAF 

March  2009 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 


AFIT/GCA/ENV/09-M05 


THE  CORRELATION  OF  HUMAN  CAPITAL  ON  COSTS  OF  AIR  FORCE 

ACQUISITION  PROGRAMS 


Stephen  E.  Gray,  BBA,  MSM 
Captain,  USAF 


Approved: 


//SIGNED//  12  Mar  09 

Lt  Col  Patrick  D.  Kee  (Chairman)  Date 

//SIGNED//  13  Mar  09 

Lt  Col  R.  David  Fass  (Member)  Date 

//SIGNED//  12  Mar  09 

Maj  Todd  A.  Peachey  (Member)  Date 


AFIT/GCA/ENV/09-M05 


Abstract 

Previous  studies  have  linked  human  capital  with  performance.  The  studies  have 
shown  performance,  measured  as  output,  of  a  firm  can  be  increased  by  adding  more 
personnel  and/or  increasing  education  levels.  This  research  uses  the  Cobb-Douglas 
Production  function  to  build  upon  that  relationship  of  inputs  to  outputs.  The  output  in 
this  study  is  the  average  cost  overrun  of  Aeronautical  Systems  Center  research, 
development,  test,  and  evaluation  contracts  managed  from  1994  to  2008.  The  inputs  are 
the  numbers  of  military/government  employee  financial  managers,  program  managers, 
engineers,  rated  personnel,  and  military/govemment  employees  in  those  career  fields 
possessing  a  graduate  degree  or  higher.  A  time  series  regression  was  conducted  using 
those  inputs  to  outputs  while  controlling  for  other  factors  such  as  budgetary  fluctuations, 
inflation,  and  individuals  doing  defense  related  work  in  the  private  sector  as  a  proxy  for 
contractors  working  in  program  offices.  The  results  indicate  a  negatively  correlated 
relationship  exists  between  human  capital  and  cost  overruns. 
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THE  CORRELATION  OF  HUMAN  CAPITAL  ON  COSTS  OF  AIR  FORCE 

ACQUISITION  PROGRAMS 


I.  Introduction 


Purpose 

The  purpose  of  this  research  was  to  determine  if  there  is  a  relationship  between 
the  number  of  program  office  personnel  and  their  corresponding  education  levels  on  cost 
variances  for  contracts  managed  by  the  Aeronautical  Systems  Center  (ASC)  located  at 
Wright  Patterson  Air  Force  Base,  Ohio  (WPAFB).  This  research  focused  on  civilian  and 
military  personnel  assigned  to  the  cost,  program  management,  and  engineering  career 
fields  as  well  as  rated  officers  assigned  to  acquisition  positions  within  the  Air  Force 
Materiel  Command  (AFMC)  at  WPAFB.  Rated  officers  are  military  personnel  that 
possess  an  aeronautical  rating. 

Previous  Research 

The  Department  of  Defense  (DoD)  routinely  experiences  difficulty  in  fielding 
major  weapon  systems  within  allotted  budgets.  The  Air  Force  is  no  exception  and  has 
historically  struggled  with  cost  overruns.  Multiple  studies  have  been  completed  with 
little  success  in  finding  a  policy  lever  to  implement  to  overcome  cost  overruns.  The  next 
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few  sections  of  this  chapter  briefly  cover  studies  completed  on  overruns,  acquisition 
reform,  the  private  sector,  the  current  Air  Force  labor  force  and  the  research  objective. 
Overrun  Studies 

Historical  studies  on  cost  overruns  indicate  they  have  ranged  from  an  underrun  of 
1%  to  an  overrun  of  68%  (Christensen  &  Gordon,  1998;  Drezner,  Jarvaise,  Hess,  Norton, 
&  Hough,  1993;  Rusnock,  2008).  The  results  from  those  studies  varied  with  respect  to 
the  causes  of  the  cost  overruns.  Drezner  et  al.  (1993)  found  there  was  no  “silver  bullet” 
strategy  in  bringing  cost  overruns  under  control.  Their  study  quantified  the  magnitude  of 
weapon  system  cost  growth  along  a  number  of  dimensions  but  they  could  not  definitively 
account  for  the  observed  cost  growth  patterns  (Drezner  et  al.,  1993).  Christensen  et  al. 
(1998)  identified  estimation  error  as  the  causal  factor  in  cost  overruns.  Rusnock  (2008) 
discovered  cost  overruns  vary  significantly  by  commodity  type.  She  found  cost  overruns 
in  space  systems  varied  significantly  from  those  in  aircraft  and  missiles  and  ascertained 
certain  predictor  variables  have  a  causal  relationship  with  overruns  in  space  systems.  She 
found  communications  missions,  ground  equipment,  firm-fixed  price  contracts,  and 
increased  program  manager  tenure  are  all  predictive  of  lower  cost  growth  for  military 
space  systems  (Rusnock,  2008).  Regardless  of  the  cause  of  the  overrun,  the  increased 
costs  result  in  additional  funds  being  directed  to  the  program  thus  creating  instability  in 
the  planning,  programming,  budgeting,  and  execution  (PPBE)  system  currently  used  by 
the  DoD.  The  perpetual  drain  on  investment  accounts  as  a  result  of  instability  was  what 
led  Dr.  Paul  G.  Kaminksi,  a  former  Under  Secretary  of  Defense  for  Acquisition,  to  adopt 
program  stability  as  his  number  one  acquisition  reform  initiative  (Czelusniak,  1997).  in 
extreme  cases,  overruns  led  to  program  cancellations  like  the  Navy’s  A- 12  program  and 
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the  Tri-Service  Standoff  Attack  Missile  (TSSAM)  for  the  Air  Force  (FAS,  2009).  Both 
programs  experienced  significant  cost  growth  leading  to  the  DoD  cancelling  them  (FAS, 
2009). 

Acquisition  Reform 

Some  historical  studies  reviewed  specifically  acquisition  refonn  and  whether  or 
not  the  initiatives  set  forth  by  the  government  had  any  effect  on  cost  overruns.  The 
results  of  those  studies  are  mixed.  For  example,  Christensen  et  al.  (1999)  examined  269 
defense  programs  from  1988  through  1995  and  found  the  Packard  Commission 
recommendations  did  not  reduce  average  cost  overruns  at  all.  During  that  time  period, 
the  cost  perfonnance  on  those  contracts  actually  worsened  significantly  (Christensen  et 
al.,  1999).  The  Packard  Commission  was  initiated  in  1985  by  President  Reagan  and  was 
focused  on  broad  organizational  changes  within  the  DoD  and  would  become  the  Reagan 
Administration’s  most  recognized  acquisition  reform  movement  (Holbrook,  2003). 
Drezner  et  al.  (1993)  found  similar  results  of  the  ineffectiveness  of  reform  initiatives  in 
reviewing  197  acquisition  programs  from  1960  through  1990.  Holbrook  (2003)  also 
found  parallel  results.  There  was  no  statistically  significant  difference  in  the  overruns 
percentages  on  pre-reform  contracts  and  post-reform  contracts  (Holbrook,  2003). 
Smirnoff  (2006)  however  found  the  Packard  Commission,  the  Nunn-McCurdy  Act  of 
1982  and  the  Federal  Acquisition  Streamlining  Act  (FASA)  of  1994  all  led  to  significant 
cost  savings.  The  Nunn-McCurdy  Act  requires  the  DoD  to  notify  Congress  when  cost 
growth  exceeds  15%  of  the  original  estimate  and  calls  for  programs  to  be  cancelled  if 
growth  is  over  25%  of  the  original  estimate  unless  the  DoD  submits  justification  for  the 
program  to  continue.  FASA’s  intent  was  to  overhaul  the  cumbersome  and  complex 
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procurement  system  of  the  federal  government  and  more  importantly  the  DoD  (Holbrook, 
2003). 

Private  Sector  Studies 

The  previous  studies  attempted  to  identify  causes  of  overruns  and  effectiveness  of 
government  wide  refonn  initiatives.  Some  private  sector  studies  go  even  further  by 
scrutinizing  individual  workers  to  detennine  the  effects  of  individual  efforts  on  overall 
firm  productivity.  The  studies  are  not  specific  to  DoD  cost  estimating  but  the  same 
outcome  is  generalizable  to  any  organization  because  the  studies  measured  individual 
efforts  without  regard  to  specific  career  fields.  The  two  types  of  studies  are  those  relating 
to  the  number  of  personnel  (Aral,  Brynjolfsson  &  Van  Alstyne,  2007;  Li,  2002;  Shaw  & 
Weekley,  1985)  and  those  relating  to  the  education  level  of  personnel  (Bartel,  1994; 
Lynch  &  Black,  1995;  Nelson  &  Phelps,  1966).  These  studies  concluded  the  number  of 
personnel  and  their  corresponding  education  level  are  positively  correlated  with  firm 
performance.  A  few  of  the  studies  argued  for  some  of  the  indirect  benefits  of  education 
for  the  firm.  Two  studies  found  the  more  educated  a  firm’s  employees  are,  the  more 
likely  they  are  to  be  innovative  and  productive  (Blundell,  Dearden  &  Meghir,  1996; 
Lynch  &  Black,  1995). 

Current  Air  Force  Manning  and  Research  Objective 

The  Air  Force  has  incurred  manpower  reductions  throughout  its  history  and  has 
lost  some  very  talented  personnel  within  its  ranks.  However,  the  Air  Force  realized 
favorable  retention  rates  following  September  11,  2001,  and  received  additional 
manpower  authorizations  from  Congress  due  to  the  global  war  on  terrorism  (GWOT) 

(. Military  Personnel  End  Strength ,  n.d.).  Those  additional  authorizations  were  on  a  year 


4 


by  year  basis.  The  Air  Force  anticipated  their  authorized  end  strength  numbers  would 
decrease  over  time  as  the  GWOT  efforts  would  hopefully  diminish  and  the  entire  DoD 
would  begin  to  downsize  once  again. 

In  order  to  meet  their  projected  end  strength  numbers,  the  Air  Force  instituted 
force  shaping  initiatives  which  allowed  and  in  some  cases  forced  service  members  to 
depart  the  Air  Force  ahead  of  their  active  duty  service  commitments.  This  decision  is 
currently  a  hot  topic  of  debate  among  Air  Force  leaders  and  members  alike.  The 
Secretary  of  Defense,  in  considering  the  burdens  created  by  a  reduced  workforce,  did  not 
think  force  shaping  was  the  right  tool  at  the  time  to  re-size  the  Air  Force  so  he  halted  the 
force  shaping  initiatives  in  June  of  2008  (Randolph,  2008)  shortly  after  forcing  the 
Secretary  of  the  Air  Force  Michael  Wynne  and  the  Chief  of  Staff  of  the  Air  Force 
General  Michael  Moseley  to  resign. 

Force  shaping  was  just  one  of  many  initiatives  in  the  history  of  the  Air  Force  to 
trim  down  the  ranks.  This  research  is  not  concerned  with  the  overarching  reasoning 
behind  reducing  the  force  but  instead  tries  to  establish  a  relationship  between  the  number 
and  education  level  of  people  in  the  Air  Force  and  cost  overruns.  The  reduction  of 
manpower  over  the  years  while  at  the  same  time  the  amount  of  acquisition  work  has 
remained  relatively  stable  or  increased  in  some  years  is  the  reason  behind  this  research. 
The  amount  of  money  spent  on  Air  Force  RDT&E  programs  theoretically  relates  to  the 
amount  of  manpower  required  in  supporting  those  programs.  In  other  words,  large  dollar 
value  RDT&E  contracts  require  many  smart  and  talented  personnel  to  oversee  them 
because  there  are  several  parts  of  that  high  dollar  RDT&E  contract  to  oversee.  In  Figure 
1  on  the  next  page,  the  Air  Force  overall  end  strength  representing  all  career  fields  has 
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steadily  decreased  while  the  Air  Force  total  RDT&E  budget  has  either  remained 
reasonably  constant  or  slightly  increased.  This  study  will  attempt  to  determine  if  adding 
more  people  or  education  will  help  with  better  managing  those  RDT&E  programs 
resulting  in  fewer  overruns. 


. Total  Air  Force  End  Strength  (in  thousands) 

- Air  Force  RDT&E  Budget  (CY09$B) 

630 
610 
590 
570 
550 
530 
510 
490 


Figure  1  Air  Force  End  Strength-includes  both  active  duty  and  civil  service 
employees  (DoD  FY09  Budget  Greenbook) 


Organization  of  the  Study 

This  chapter  established  the  incentive  for  this  study  and  the  research  objective. 
Chapter  2  provides  a  review  of  the  literature  related  to  this  research.  Chapter  3  details  the 
methodology  used  to  model  the  relationship  between  human  capital  and  cost  overruns 
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and  Chapter  4  provides  the  results  of  the  analysis  with  a  brief  discussion  of  the  significant 
findings.  Lastly,  Chapter  5  provides  conclusions  on  the  study  and  recommends  future 
areas  of  research. 
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II.  Literature  Review 


This  research  examines  the  relationship  between  cost  overruns  and  human  capital. 
Specifically  it  examines  the  education  and  number  of  personnel  assigned  to  career  fields 
that  make  up  the  bulk  of  the  composition  of  program  offices  at  WPAFB  on  the  cost 
variances  of  contracts  managed  by  ASC.  This  chapter  provides  a  background  on 
previous  literature  that  has  been  published  on  this  subject. 

Historical  Reform  in  Acquisitions 

We  expect  to  achieve  greater  successes  from  every  person,  dollar,  and  hour  we  expend  to 
acquire  and  sustain  our  current  and  new  weapon  systems. 

-Darlene  Druyun,  Principal  Deputy  Assistant  Secretary  of  the  Air  Force 
for  Acquisition  and  Management  1997 

There  have  been  multiple  studies  and  audits  regarding  DoD  acquisitions  which  I 
will  briefly  cover  in  this  chapter.  Researchers  and  decision  makers  are  always  looking 
for  a  policy  lever  in  acquisition  refonn  to  try  to  control  costs  and  schedule. 

Unfortunately,  the  results  are  inconclusive  as  to  what  that  policy  lever  might  be.  This 
section  briefly  reviews  some  historical  actions  and  reforms  undertaken  by  the  DoD  to 
control  costs.  In  the  early  1990s,  senior  leaders  within  the  Department  of  the  Air  Force 
recognized  the  need  to  streamline  acquisitions  and  become  more  efficient  due  to  the 
shrinking  DoD  industrial  base  and  budget.  Some  examples  of  historical  studies  initiated 
by  the  government  are  the  Blue  Ribbon  Commission  on  Defense  Management  (Packard 
Commission)  of  1986  to  review  the  consolidation  of  the  defense  industrial  base. 

Resulting  reforms  include  the  Federal  Acquisition  Streamlining  Act  of  1994  to  revise  the 
DoD  5000  series  regulations  incorporating  flexibility  into  acquisition  policies  and 
procedures  (Holbrook,  2003).  In  addition  to  the  above  quote  by  Ms.  Druyun  and  putting 
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aside  any  irony  that  would  later  develop  from  her  statement,  she  goes  on  to  say  the 
necessary  reform  direction  .  .is  basically  toward  creating  a  partnership  with  our 
contractors.  They  are  not  our  enemy.  If  we  erect  a  wall  between  us,  then  chances  are  we 
are  going  to  walk  away  with  a  failure”  (Kitfield,  1997).  There  was  a  huge  movement 
within  the  Air  Force  to  get  contracts  under  control  and  to  rein  in  the  contractors  on 
perfonnance,  costs,  and  schedule.  One  initiative  that  came  about  during  this  timeframe 
was  an  approach  called  TSPR  (Total  System  Perfonnance  Responsibility).  TSPR  is  an 
acquisition  approach  that  responds  to  the  government’s  and  industry’s  recognition  of 
change  needed  in  government  procurement  (Pandes,  2001).  TSPR  was  a  creative 
contracting  method  in  that  it  moved  several  of  the  functions  previously  accomplished  by 
the  government  side  to  the  contractor.  TSPR  was  the  response  to  another  Secretary  of 
Defense  initiative  requiring  the  use  of  performance  specifications  rather  than  military 
specifications  (Pandes,  2001). 

Additionally,  there  were  several  reforms  in  Earned  Value  Management  Systems 
(EVMS)  during  the  early  1990s.  Earned  value  is  a  measurement  tool  used  to  manage 
projects.  It  does  this  by  tracking  the  perfonnance  of  work  units  in  the  work  break  down 
structure,  and  tracking  cost  and  schedule  of  work  performed  against  a  planning  baseline 
((Defense  Acquisition  University,  2008).  Earned  value  programs  consisting  of  32 
stringent  criteria  have  been  in  place  for  large,  flexibly  priced  defense  contracts  since  1967 
(Christensen,  1998).  Originally,  the  criteria  were  established  by  the  Air  Force  as 
cost/schedule  planning  and  control  specifications  (C/SPEC)  but  they  were  later  adopted 
by  all  the  military  services  as  cost/schedule  control  systems  criteria  (C/SCSC).  C/SCSC 
was  then  slightly  modified  and  renamed  earned  value  management  control  systems 
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criteria,  all  the  while  maintaining  the  principles  of  the  32  original  criteria  (Christensen, 
1998).  Over  the  years,  implementation  of  the  criteria  consistently  across  the  services  or 
even  within  the  same  service,  proved  to  be  a  cumbersome  project.  Some  studies 
indicated  the  criteria  was  more  non-value  added  activity  by  contractors  and  program 
managers  (GAO,  1997).  The  fundamental  use  of  earned  value  systems  was  tainted  when 
the  Navy’s  A-12  program  was  cancelled.  The  shortcomings  of  the  system  and  those  who 
used  it  were  notoriously  cited  as  potential  reasons  for  the  downfall  of  the  A-12.  When 
Secretary  of  Defense  Dick  Cheney  canceled  the  program  in  January  1991,  he  complained 
that  no  one  could  tell  him  its  final  cost  (Morrison,  1991).  Earned  value  concepts  are 
integral  to  the  management  of  large,  complex  acquisition  efforts  and  if  used  objectively, 
can  facilitate  successful  project  completion. 

Reformation  of  regulations,  modification  to  acquisition  processes,  and 
implementation  of  systems  are  all  active  approaches  in  trying  to  control  contract 
performance.  The  quantity  and  quality  of  acquisition  personnel  should  also  be  reviewed 
as  a  feasible  alternative  to  managing  DoD  contracts  because  the  personnel  are  just  as 
important  as  the  regulations  and  processes.  Military  officers  that  are  financial  managers 
routinely  seek  positions  outside  of  acquisitions  because  the  acquisition  corps  is 
sometimes  considered  a  career  killer  for  the  finance  community  and  it  can  be  difficult  to 
attain  higher  rank  while  remaining  assigned  to  acquisitions  (McCrary,  2008).  In  2009, 
the  United  States  House  of  Representatives  approved  a  2009  authorization  bill  with 
provisions  proposing  sweeping  changes  to  the  way  the  Pentagon  acquires  major  weapon 
systems.  One  of  the  many  changes  proposed  in  the  bill  is  the  creation  of  an  acquisition 
career  track  within  the  military,  including  general  officer  positions  in  each  service, 
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agency,  and  combatant  command  (Bennett,  2008).  Many  studies  have  linked  acquisition 
problems  to  the  shrinking  acquisition  corps  among  the  many  other  factors  discussed 
previously  in  this  section  (Holbrook,  2003;  Feuring,  2007). 

Cost  Variance 

There  are  two  types  of  cost  variance  measurements  (Feuring,  2007).  One 
measure,  cost  growth,  compares  a  program’s  beginning  budget  versus  the  final  budget  to 
see  if  there  was  an  increase  in  the  overall  costs.  The  other  measure,  cost  overrun, 
analyzes  the  cost  performance  of  a  program’s  contract  to  see  if  there  was  a  difference  in 
the  budgeted  cost  versus  the  actual  cost  at  several  points  throughout  the  period  of 
performance  of  the  contract.  Sipple,  White  and  Greiner  (2004)  reviewed  a  myriad  of 
studies  on  cost  overruns  and  indicated  the  causes  of  those  overruns  are  the  starting  point 
to  identify  a  model  that  allows  cost  variance  to  be  predicted  and  controlled  (Feuring, 
2007).  Feuring’s  (2007)  focused  research  on  cost  overruns  indicated  they  can  be  reduced 
by  adding  more  cost  analysts  or  increasing  education  levels  of  existing  cost  analysts. 

This  work  expands  on  Feuring’s  work  and  adds  program  managers,  engineers,  and  rated 
personnel  as  having  a  correlating  relationship  with  cost  overruns. 

Several  studies  indicate  estimation  errors  as  a  common  cause  of  both  cost 
overruns  and  cost  growth.  Christensen,  Searle,  and  Vickery  (1999)  found  costs  are 
underestimated  initially,  specifically  Milestone  B  or  the  “official  start”  of  an  acquisition 
program,  which  biases  the  remaining  estimates  in  many  cases.  Subsequent  estimates  use 
the  initial  estimate  as  a  baseline  so  it  is  difficult  to  recover  from  a  “bad  baseline”  in 
building  an  accurate  final  program  cost.  As  a  program  requires  more  funds  than  were 
originally  planned,  decision  makers  have  to  respond  by  either  reducing  the  quantity  or 
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performance  for  the  system,  or  they  have  to  reallocate  funds  from  other  programs  in  order 
to  make  up  for  the  shortfall  (Arena  et  ah,  2006).  Reallocating  from  other  programs  or 
cutting  requirements  from  the  existing  program  only  exacerbates  the  problem  across  the 
entire  resource  constrained  Air  Force  (Czelusniak,  1997).  These  decisions  eventually 
lead  to  more  bad  decisions  resulting  in  increased  costs.  Estimation  error  not  only  occurs 
up  front,  but  can  be  found  throughout  the  performance  of  a  contract,  especially  if 
requirements  are  constantly  changing  and  the  budget  is  unstable.  Pannell  (1994)  found 
many  of  the  estimation  errors  occur  during  the  production  periods  of  acquisition 
programs.  These  errors  in  the  estimation  of  production  costs  arise  because  of  cost 
estimating  errors,  data  omissions,  schedule  slips  attributable  to  technical  problems, 
system  weight  growth,  and  inadequately  scoped  engineering  efforts  (Pannell,  1994).  The 
Office  of  the  Secretary  of  Defense  Cost  Analysis  Improvement  Group  (OSD  CAIG, 

2001)  reviewed  several  programs  trying  to  uncover  the  causes  of  cost  variances  and  they 
detennined  two  main  effects  were  to  blame:  mistakes  and  decisions.  Mistakes  such  as 
underestimated  efforts  and  decisions  such  as  using  too  optimistic  of  a  learning  curve  were 
specifically  mentioned  as  causes  of  error. 

Decisions  and  the  Role  of  Systems  Engineering 

There  are  obviously  more  key  players  in  the  process  than  just  cost  analysts  that 
make  poor  estimating  decisions  resulting  in  cost  overruns.  A  typical  program  office  is 
manned  by  several  different  specialties.  This  research  focuses  on  a  select  few  from  the 
government  side  because  ultimately,  they  are  the  key  players  in  the  decision  making 
process.  This  study  specifically  examines  program  managers,  engineers,  rated  officers, 
and  cost  analysts  assigned  to  WPAFB  working  on  aircraft  development  programs.  All 
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four  areas  must  work  closely  together  to  keep  a  program  on  schedule  and  on  cost  while 
maintaining  the  requirements  desired  by  the  end  user.  The  program  manager  is 
ultimately  responsible  for  making  the  decisions,  but  they  must  rely  heavily  upon  the  team 
of  engineers  for  technical  requirements,  the  cost  analysts  for  fiscal  requirements,  and  the 
rated  officers  for  end  user  performance  requirements.  In  many  cases,  rated  officers  are 
also  program  managers  at  WPAFB.  The  rated  officers  serving  as  program  managers  have 
dual  responsibilities  in  that  they  must  make  sure  the  performance  requirements  of  the  end 
user  are  being  met  while  also  managing  cost  and  schedule  requirements  to  ensure  the 
program  can  succeed.  The  process  of  managing  programs  has  been  studied  extensively 
by  the  DoD.  Numerous  research  efforts  have  been  completed  looking  for  policy  levers  in 
executing  successful  acquisition  management.  Regulations  have  even  been  written  to 
establish  a  simplified  and  flexible  management  framework  for  translating  mission  needs 
into  stable,  affordable,  and  well-managed  programs  (DoD  5000.2,  2003).  The  effective 
integration  of  the  four  previous  mentioned  career  fields’  contributions  are  paramount  to 
the  success  of  the  overall  acquisition  system.  The  acquisition  system  is  an  integrated 
composite  of  people,  products,  and  processes  that  provide  a  capability  to  satisfy  a  stated 
need  or  objective  (DAU  SE  Handbook,  2001).  All  four  key  specialties  must  be  well 
versed  in  DoD  acquisitions  and  understand  how  they  each  play  a  vital  role  in  the  overall 
system.  In  1990,  Congress  recognized  that  importance  and  enacted  the  Defense 
Acquisition  Workforce  Improvement  Act  (DAWIA,  1990).  Congress  intended  for 
DAWIA  to  improve  the  effectiveness  of  the  personnel  who  manage  and  implement 
defense  acquisition  programs  (Garcia  et  al.,  1997).  All  personnel  assigned  to  acquisition 
positions  must  achieve  and  maintain  certification  levels  for  their  field,  as  required  by  the 
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DAWIA.  One  of  the  requirements  for  certification  is  the  completion  of  Acquisition  101 
(ACQ  101),  Fundamentals  of  System  Acquisition  (DAU,  2008).  In  ACQ  101,  all  of  the 
key  players  in  acquisitions  are  introduced  and  corresponding  roles  they  play  in  the 
acquisition  process  are  explained.  A  key  component  explained  in  ACQ  101  is  the 
concept  of  systems  engineering  management.  As  illustrated  by  Figure  2,  systems 
engineering  management  is  accomplished  by  integrating  three  major  activities: 
development  phasing  that  controls  the  design  process  and  provides  baselines  that 
coordinate  design  efforts,  a  systems  engineering  process  that  provides  a  structure  for 
solving  design  problems  and  tracking  requirements  flow  through  the  design  effort,  and 
life  cycle  integration  that  involves  customers  in  the  design  process  and  ensures  the 
system  developed  is  viable  throughout  its  life  (DAU  SE  Handbook,  2001). 


Figure  2  Systems  Engineering  Activities  (DAU  SE  Handbook,  2001) 


Throughout  the  systems  engineering  process  decisions  are  made  by  personnel  in  all  four 


areas  of  expertise.  Some  examples  include  but  are  not  limited  to: 
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•  The  cost  analyst  must  decide  on  ground  rules  and  assumptions  in  estimating  the 
costs  based  upon  best  available  data. 

•  The  engineers  must  decide  on  acceptable  perfonnance  parameters  and  technology 
readiness  levels. 

•  The  rated  officers  bring  expertise  and  credibility  for  dealing  with  end  users  in 
communicating  the  user  needs. 

•  The  program  manager  must  decide  on  schedule  baselines  and  act  upon  the 
decisions  of  the  other  three  key  players  in  the  process. 

An  important  distinction  to  note  here  is  all  of  the  decisions  are  interrelated.  That  is,  no 
one  particular  decision  should  be  made  without  considering  the  effects  of  that  decision  on 
the  other  areas.  This  concept  revolves  around  effective  communication.  The  foundation 
of  systems  engineering  is  communication  and  it  is  imperative  to  program  success  that  the 
channels  of  communication  are  wide  open  and  clear  (DAU  SE  Handbook,  2001).  The 
concept  of  systems  engineering  should  not  consume  the  overall  efforts  of  the  program. 
However,  key  decision  makers  should  be  well  versed  in  the  concept.  An  analysis  by  the 
International  Council  on  Systems  Engineering  (INCOSE)  Systems  Engineering  center  of 
excellence  (SECOE)  indicates  that  optimal  effort  spent  on  Systems  Engineering  is  about 
15-20%  of  the  total  project  effort  (Honour,  2002).  At  the  same  time,  studies  have  shown 
that  Systems  Engineering  essentially  leads  to  reduction  in  costs  among  other  benefits 
(Honour,  2002). 
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Aircraft  Programs  Versus  Other  Programs 

Previous  research  on  cost  overruns  was  at  a  macro  level  and  has  either  looked 
across  the  entire  DoD  (Holbrook,  2003)  or  the  entire  Air  Force  (Feuring,  2007).  This 
study  takes  a  more  micro  approach  similar  to  Rusnock  (2008).  Rusnock  looked  at 
programs  managed  by  the  Air  Force  Space  Command  (AFSPC)  whereas  this  research 
looks  at  programs  managed  by  ASC,  primarily  aircraft  programs.  While  Rusnock’s 
research  focused  on  cost  growth,  our  research  will  focus  on  cost  overruns.  The  Air  Force 
Material  Command  (AFMC)  manages  three  product  centers:  ASC,  Air  Armament  Center, 
and  Electronic  Systems  Center  (ESC)  (AFMC  Factsheet,  2008).  While  all  three  product 
centers  manage  different  types  of  programs,  the  results  of  this  research  are  generalizable 
to  the  other  centers  as  well  because  the  acquisition  profiles  of  all  three  centers  are 
relatively  the  same.  The  reasoning  behind  looking  only  at  ASC  managed  programs  was 
threefold:  First,  the  ASC  data  is  easily  available;  second,  the  close  proximity  of  ASC;  and 
third,  the  acquisition  profile  of  space  is  quite  different  than  aircraft. 

Military  space  systems  differ  from  other  defense  systems  in  two  ways:  their 
operational  environment  and  their  acquisition  life  cycle  (Rusnock,  2008).  Space  vehicles 
operate  in  much  harsher  environments  than  those  of  aircraft.  Those  environments  are 
also  very  remote,  often  thousands  of  miles  up  in  space.  Corrections  or  alterations  to  the 
on-orbit  system  range  from  extremely  expensive  at  best,  to  virtually  impossible.  Space 
systems  are  also  frequently  acquired  in  small  quantities  as  compared  to  aircraft  and 
follow  on  maintenance  for  launched  systems  is  practically  non-existent.  Because  of  these 
factors,  heavy  emphasis  is  placed  upon  the  survivability  of  the  system  early  on  in  the 
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acquisition  life  cycle  and  few  dollars  are  spent  in  the  production  and  sustainment  phases 
as  compared  to  the  RDT&E  phase  (Rusnock,  2008)  as  compared  to  aircraft  programs.  As 
seen  in  Figure  3,  an  aircraft  system  incurs  most  of  its  costs  in  the  operating  and  support 
phase  whereas  Figure  4  shows  a  space  system  incurs  the  majority  of  their  costs  during  the 
systems  acquisition  phase.  The  difference  in  cost  distribution  is  the  result  of  the 
significant  costs  of  developing  the  space  vehicle  and  launching  it  which  is  paid  with 
RDT&E  funds.  In  comparison,  the  O&M  costs  are  relatively  minor  and  represent  the 
resources  needed  to  pay  for  ground  support  operations.  Our  research  specifically 
examines  RDT&E  cost  plus  contract  cost  overruns  so  it  would  not  be  appropriate  to 
include  space  system  acquisitions. 
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Figure  3  Aircraft  System  Life  Cycle  Cost  Curve  (Rusnock,  2008) 
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Figure  4  Space  System  Life  Cycle  Cost  Curve  (Rusnock,  2008) 
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It  is  worth  mentioning  software  development  programs  are  also  quite  different 
from  aircraft  and  space  systems  which  are  why  we  did  not  include  those  types  of  efforts 
in  our  research.  Software  acquisition  has  steadily  increased  since  the  early  1980s  where 
software  development  comprises  a  large  portion  of  just  about  any  development  effort. 
Analysts  historically  determine  costs  based  on  the  number  of  Source  Lines  of  Code 
(SLOC).  Estimates  differ  depending  on  how  many  SLOC  are  commercial  off  the  shelf 
(COTS)  versus  originally  authored  by  programmers.  The  total  actual  SLOC  number, 
however,  will  likely  be  greater  as  past  experience  with  software  estimation  has  shown  the 
size  is  typically  both  underestimated  and  functionality  is  added  as  the  development 
progresses  (Pracchia,  2004).  That  is  not  to  say  aircraft  program  requirements  remain 
stable.  Compared  to  aircraft  programs  which  are  approaching  100  years  of  development, 
software  development  is  a  rapidly  changing  environment  and  technology  often  outpaces 
development  efforts  leading  to  many  unknowns  in  estimating  software  costs.  A  software 
cost  estimation  model  is  doing  well  if  it  can  estimate  software  development  costs  within 
20%  of  the  actual  costs,  70%  of  the  time,  and  within  the  class  of  projects  to  which  it  has 
been  calibrated  (Boehm,  1984). 

Human  Capital 

The  sheer  number  of  people,  their  corresponding  training  or  education  in  the  field 
in  which  they  work,  and  their  motivation  all  affect  job  perfonnance  and  efficiency 
(Bartel,  1994;  Lynch  &  Black,  1995).  Feuring  (2007)  found  education  of  cost  analysts  to 
be  statistically  significant  on  the  amount  of  cost  overruns  in  the  Air  Force.  There  are 
several  differences  between  education  and  training  but  due  to  data  limitations,  this 
research  will  focus  on  the  education  of  individuals.  Acquisition  specific  training  via  the 
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Defense  Acquisition  University  (DAU)  is  highly  regarded  within  the  acquisition 
community  and  is  required  for  professionals  assigned  to  acquisitions  (DAWIA,  1990). 
There  are  specific  Acquisition  Professional  Development  Program  (APDP)  levels  that 
must  be  attained  at  certain  points  throughout  acquisition  careers  (DAWIA,  1990).  Again, 
due  to  data  limitations,  that  workforce  specific  ADPD  training  data  is  not  available.  As  a 
result,  this  research  will  focus  on  post  secondary  education  levels.  Previous  research 
supports  that  more  highly  educated  individuals  have  a  greater  probability  of  receiving 
both  employer-provided  training  and  work-related  training  (Blundell  et  ah,  1996).  Finns 
that  invest  in  employees’  education  tend  to  see  returns  on  those  investments  either  in  the 
form  of  increased  productivity  or  employee  loyalty  to  the  finn  (Bartel,  1994). 

Companies  that  retain  highly  skilled  workers  will  nonnally  perfonn  better  than  those 
companies  that  either  do  not  incentivize  education  or  allow  their  educated  employees  to 
walk  out  the  door  (Bartel,  1994).  The  next  section  will  look  at  previous  research  on  the 
number  of  personnel  in  a  finn  while  the  second  section  deals  with  education  levels  of 
employees. 

Number  of  People 

Feuring’s  (2007)  research  concluded  that  increasing  the  number  of  cost  analysts 
can  have  a  decreasing  effect  on  cost  overruns.  Teams  usually  outperfonn  individuals  in 
just  about  any  task  (Hill,  1982).  Teams  made  up  of  the  right  skill  set  and  quantity  of 
people  will  certainly  outperform  individuals  (Bliss  &  Potter,  2005).  Bliss  and  Potter’s 
(2005)  research  found  that  in  a  sample  of  over  three  thousand  mutual  funds  over  a  twelve 
year  horizon,  funds  managed  by  teams  outperformed  individual  managed  funds  by  over 
70  basis  points  per  year  and  had  lower  expense  ratios.  Even  the  fastest  and  smartest 
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individuals  have  limits  and  their  perfonnance  will  start  to  decline  once  that  saturation 

point  has  been  reached.  For  example,  a  stock  analyst’s  performance  begins  to  decline 

once  the  portfolio  they  are  managing  exceeds  12-13  stocks  (Li,  2002).  Another  study 

found  an  inverted-U  shaped  relationship  exists  between  multitasking  and  productivity 

such  that,  beyond  an  optimum,  more  multitasking  is  associated  with  declining  project 

completion  rates  and  revenue  generation  (Aral  et  ah,  2007).  Some  research  has 

considered  that  optimum  for  two  of  the  variables  we  are  examining.  Wheelwright  & 

Clark  (1992)  explained  the  following  for  engineers: 

When  an  engineer  focused  on  a  single  project  is  given  a  second  one,  utilization 
often  rises  slightly  because  the  engineer  no  longer  has  to  wait  for  the  activities  of 
others  involved  in  that  single  project.  Instead,  the  engineer  can  move  back  and 
forth  between  the  two  projects.  However,  if  a  third,  fourth,  or  even  fifth  project  is 
added,  the  percentage  of  time  spent  on  value-adding  tasks  drops  rapidly,  as  an 
increasing  fraction  of  valuable  time  is  spent  on  non  value-adding  tasks— 
coordinating,  remembering,  or  tracking  down  infonnation,  for  example,  (p.  90) 

Feuring  (2007)  found  that  as  defense  employment  grew,  cost  overruns  actually  increased. 

As  the  Air  Force  gets  bigger,  we  incur  more  overruns  because  the  Air  Force  has  become 

too  large  and  has  entered  into  an  area  of  diseconomies  of  scale  (Feuring,  2007).  These 

results  could  be  generalized  to  a  typical  program  office  where  complex  systems  are  being 

managed  and  critical  decisions  are  made  by  team  members.  There  are  also  indirect 

aspects  of  being  overworked.  Workers  will  begin  to  resent  their  jobs  and  with  the 

increase  in  anxiety  and  stress,  their  performance  may  begin  to  falter.  In  an  experimental 

study,  workers  in  overloaded  situations  felt  more  pressure,  had  less  task  enjoyment,  and 

felt  more  depressed  (Shaw  &  Weekley,  1985).  If  this  sort  of  situation  were  to  be 

prolonged,  one  might  expect  it  to  have  a  negative  effect  on  performance  (Feuring,  2007). 
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Education  Levels  of  Employees 

Cost  overruns  cannot  be  corrected  by  just  adding  more  people  to  program  offices. 
Program  offices  may  already  have  the  right  number  of  people  but  they  may  not  have  the 
right  mix  of  people  of  the  proper  number  of  educated  people.  So,  adding  more  education 
to  those  personnel  assigned  to  program  offices  may  be  a  part  of  the  solution.  Previous 
research  examined  education  at  both  the  individual  and  firm  level,  but  this  research  will 
focus  only  on  productivity  at  the  firm  level.  Program  offices  are  teams  of  people  doing 
acquisition  efforts  so  it  would  not  be  appropriate  to  analyze  output  at  the  individual  level, 
but  more  appropriate  at  the  firm  level,  or  program  office  level.  Program  offices  typically 
consist  of  several  people  assigned  to  teams  working  toward  the  same  goal  of  acquiring 
and  delivering  weapon  systems.  Bartel  (1994)  used  a  Cobb-Douglas  production  function 
to  estimate  the  impact  of  employee  training  at  the  firm  level  because  previous  research 
had  only  focused  on  the  individual  level  (Feuring,  2007).  Organizations  that  were 
operating  below  expected  productivity  levels  increased  their  performance  after  training 
programs  were  implemented  (Bartel,  1994).  By  using  the  Cobb-Douglas  production 
again,  for  every  10%  increase  in  average  fonnal  education  levels  of  employees,  an  8.5% 
increase  in  productivity  in  manufacturing  and  12.7%  in  non-manufacturing  firms  results 
(Lynch  &  Black,  1995).  Human  capital  in  the  form  of  education  is  an  integral  component 
to  any  organization.  This  is  especially  true  in  the  ASC  organization  where  development 
of  new  aircraft  weapon  systems  include  copious  amounts  of  new  technologies.  Due  to 
those  cutting  edge  technologies,  ASC  needs  motivated  and  incredibly  smart  people. 
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Highly  educated  workers  have  a  comparative  advantage  with  respect  to  the  adjustment  to 
and  implementation  of  new  technologies  (Bartel,  1987).  Bartel  (1987)  also  states  the 
relative  demand  for  educated  workers  declines  as  the  ages  of  plant  and  equipment 
increase,  especially  in  research  and  development  intensive  industries.  Education 
enhances  one’s  ability  to  receive,  decode,  and  understand  information  (Nelson  &  Phelps, 
1966).  Due  to  the  Air  Force  constantly  updating  and  modernizing  their  weapon  systems, 
the  ability  to  handle  new  information  is  critical. 

Summary 

This  chapter  examined  previous  literature  regarding  historical  refonn  in 
acquisitions  and  the  role  of  human  capital  in  acquisitions.  The  next  chapter  will  build 
upon  those  relationships  in  constructing  a  model  to  test  the  effectiveness  of  acquisition 
personnel  and  cost  overruns. 
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III.  Methodology 


This  study  will  link  human  capital  to  performance  using  an  objective  measure  of 
performance.  The  objective  measure  will  be  the  amount  of  cost  overruns  realized  per 
year  by  all  programs  managed  by  ASC.  The  method  of  analysis  for  this  study  is  the 
Cobb-Douglas  production  function  similar  to  Feuring  (2007).  This  research  expands 
upon  Feuring’s  work  and  analyzes  additional  independent  variables  along  with  similar 
variables  used  in  his  2007  thesis.  The  next  section  will  clarify  how  the  objective  yearly 
cost  overruns  were  quantified. 

Objective  Measure  of  Performance 

There  are  multiple  measures  available  for  attempting  to  quantify  successful 
acquisition  contract  perfonnance.  A  complete  and  successful  demonstration  of  the  end 
state  system  being  procured  is  ultimately  the  goal  but  there  are  key  indicators  transparent 
throughout  the  process  to  measure  whether  or  not  the  program  is  on  an  executable  path. 
One  such  measure  is  the  schedule  of  the  project.  Are  key  milestones  being  met  along  the 
way  and  important  deliverables  occurring  in  accordance  with  the  schedule?  Many 
program  offices  spend  immeasurable  amounts  of  efforts  in  managing  schedules  and 
ensuring  the  acquisition  is  proceeding  as  planned.  Another  factor  that  receives  a  lot  of 
scrutiny  is  cost.  There  are  two  methods  in  measuring  cost  increases  and  decreases, 
specifically  cost  growth  and  cost  overrun.  Cost  growth  is  calculated  by  subtracting  the 
initial  estimate  from  the  final  cost,  while  a  cost  overrun  is  computed  by  subtracting  the 
budgeted  cost  of  work  performed  from  the  actual  cost  of  work  performed  (Christensen, 
1998).  This  cost  overrun  definition  is  consistent  with  the  Earned  Value  Management 
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System  (Defense  Acquisition  University,  2008).  This  research  is  concerned  with 
estimation  error,  so  cost  overrun  is  the  more  appropriate  measure  for  the  context  of  this 
research.  Factors  which  are  out  of  the  program’s  control,  such  as  funding  instabilities 
and/or  changes  in  the  scope  of  a  requirement,  can  lead  to  cost  growth  but  not  to  cost 
overruns  (Feuring,  2007).  The  actual  cost  of  work  perfonned  versus  the  budgeted  cost  of 
work  performed,  or  the  cost  overrun  calculation,  better  captures  how  well  a  program  was 
estimated.  Figure  5  illustrates  the  cost  variance  computation  which  in  due  course  results 
in  cost  overruns. 


Figure  5  Earned  Value  Management  Gold  Card  (DAU,  2008) 

EAC:  Estimate  at  Completion  ACWP:  Actual  Cost  of  Work  Performed 

TAB:  Total  Allocated  Budget  BCWS:  Budgeted  Cost  of  Work  Scheduled 

BAC:  Budget  at  Completion  BCWP:  Budgeted  Cost  of  Work  Performed 

PMB:  Program  Management  Baseline 
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The  ACWP  line  or  the  actual  cost  of  work  performed  reveals  the  actual  cost  of 
work  accomplished.  The  BCWS  line  or  the  budgeted  cost  for  work  scheduled  reflects  the 
value  of  work  planned  to  be  accomplished.  The  BCWP  line  or  the  budgeted  cost  for 
work  performed  is  the  value  of  work  actually  accomplished  or  the  earned  value.  The 
BCWP  takes  into  account  changes  in  the  scope  of  requirements  and  budget.  The  cost 
variance  is  computed  by  taking  the  difference  between  ACWP  and  BCWP.  The  cost 
variance  in  essence  is  the  delta  between  what  has  been  spent  and  what  was  planned  to  be 
spent  for  the  same  amount  of  work  perfonned.  Cost  variance  is  equivalent  to  a  cost 
overrun  because  it  takes  into  account  the  budgeted  amount  which  includes  program 
changes  and  changes  in  quantity  for  example  (Feuring,  2007).  In  contrast,  cost  growth 
alternatively  is  the  difference  between  ACWP  and  the  initial  budget.  Program  changes 
like  quantity  reductions  or  increased  scope  of  the  project  are  updated  in  the  BCWP  so  to 
measure  the  difference  in  the  initial  budget  and  the  BCWP  or  ACWP  would  not  reflect 
any  estimation  inaccuracies. 

Data  and  Variables 
Dependent  Variable 

We  collected  data  on  the  average  cost  overruns  of  ASC  managed  acquisitions  per 
year  from  1994  to  2008  using  cost  performance  report  (CPR)  data  from  the  Defense 
Acquisition  Executive  Summary  (DAES).  The  DAES  included  all  programs  from  the  Air 
Force  so  to  examine  ASC  specifically  the  data  was  sorted  by  the  Department  of  Defense 
Activity  Address  Code  (DoDAAC).  The  DoDAAC  is  a  code  that  uniquely  identifies  a 
unit  with  authority  to  requisition  and  receive  materiel.  ASC’s  DoDAAC  is  F33657  so 
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only  contracts  with  that  DoDAAC  were  included.  The  amount  of  overrun  for  each 
weapon  system  was  consolidated  by  fiscal  year  to  give  the  average  overrun  for  that  year 
in  percentage  terms. 


%Overrun  =  (ACWP-BCWP)  *  1 00  ( 1 ) 

(BCWP) 

Independent  Variables 

We  collected  the  number  of  individuals  working  in  program  offices  and  their 
corresponding  education  levels  from  the  Air  Force  Personnel  Center’s  Interactive 
Demographic  Analysis  System  (IDEAS)  database.  The  bulk  of  ASC  program  office 
manning  that  influences  contract  decisions,  with  respect  to  cost,  requirements,  and 
schedules  for  example,  come  from  eleven  specific  Air  Force  occupational  specialty  codes 
so  this  research  focused  on  those  respective  career  fields.  The  occupational  specialty 
codes  and  a  brief  description  of  their  duties  are  illustrated  in  figure  6. 


Occupational  Code 

Duties 

Area  of  Expertise 

65  F 

Financial  Analyst 

Financial  Management 

65W 

Cost  Analyst 

501 

Financial  Management 

63A 

Program  Manager 

Program  Management 

1101 

Program  Manager 

62E 

Engineer 

Engineering 

801 

General  Engineer 

830 

Mechanical  Engineer 

855 

Electrical  Engineer 

861 

Aerospace  Engineer 

Rated 

Operators 

Rated  Personnel 

Figure  6  Air  Force  Occupational  Codes 


The  eleven  specific  codes  were  consolidated  by  area  of  expertise  into  four  basic 
groups:  financial  management,  program  management,  engineering,  and  rated  personnel. 
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We  only  considered  personnel  assigned  to  WPAFB  owned  by  AFMC  in  the  IDEAS 
database.  It  is  important  to  note  the  database  query  included  personnel  assigned  to 
AFMC  headquarters  and  the  Air  Force  Research  Laboratory  (AFRL).  However, 
personnel  assigned  to  both  of  those  organizations  support  ASC  managed  acquisitions. 

This  research  attempted  to  verify  Feuring’s  (2007)  thesis  which  identified  a  statistically 
significant  relationship  between  cost  analysts  and  cost  overruns.  However,  due  to 
limitations  of  the  data,  our  research  combines  cost  analysts  and  financial  analysts  into  the 
same  independent  variable,  mainly  due  to  the  inclusion  of  the  civilian  financial  analyst 
career  field.  Civilian  financial  analysts  and  cost  analysts  both  have  the  same 
occupational  specialty,  501,  so  to  avoid  incorrect  statistical  inferences  with  respect  to  cost 
analysts  versus  financial  analysts,  the  two  were  combined.  The  rated  personnel  variable 
includes  both  pilots  and  navigators  in  the  ranks  of  Lieutenant  Colonel  and  below. 

The  education  level  variable  is  a  raw  percentage  of  all  the  personnel  discussed  in 
the  previous  paragraph  having  a  graduate  degree  or  higher. 

Control  Variables 

We  added  control  variables  to  the  model  to  account  for  other  plausible  causes  of 
cost  overruns.  We  included  the  historical  ASC  research  and  development  (R&D)  budget 
per  year  as  a  technological  proxy.  One  of  the  common  criticisms  of  inaccurate  estimates 
is  that  new  weapon  systems  are  becoming  more  complex  requiring  more  R&D  funds  and 
thus  are  more  difficult  to  estimate  (Feuring,  2007).  Increased  R&D  funding  could  be  the 
result  of  more  programs  or  just  more  complex  systems  so  there  is  a  possibility  for  some 
error  here.  We  want  to  account  for  other  plausible  causes  of  cost  overruns  by  tracking 
the  changes  in  R&D  funding  but  without  knowing  if  the  changes  resulted  from  more 
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complex  programs  or  just  more  programs,  we  cannot  make  statistical  significant 
inferences  to  cost  overruns  based  upon  the  changes  in  the  R&D  budget. 

Another  variable  added  to  the  model  is  the  amount  of  outsourced  work  to  private 
contractors  at  ASC.  Over  the  years,  the  amount  of  contractor  support  in  program  offices 
has  increased  substantially,  but  trying  to  quantify  that  outsourced  work  is  extremely 
difficult.  The  type  of  work  that  was  outsourced  is  even  more  difficult  to  quantify  as  these 
contractors  support  the  occupations  explored  in  this  research  as  well  as  logistics  and 
administration.  Following  the  same  method  as  Feuring’s  (2007)  model,  a  raw  number  of 
people  doing  defense  related  work  in  private  industry  was  added  to  the  model  (DoD 
Greenbook,  2009).  This  number  provides  a  trend  for  the  amount  of  work  that  was 
outsourced  per  year  by  the  DoD  so  the  same  results  could  be  generalized  to  ASC. 

We  added  unexpected  inflation  as  another  control  variable  to  the  model.  The 
budgets  executed  by  ASC  over  the  years  represent  billions  of  dollars  so  even  a  minor 
change  in  the  anticipated  inflation  represents  significant  cost  overruns.  Table  5-10  of  the 
OUSD  Greenbooks  for  various  fiscal  years  were  utilized  to  capture  the  unexpected 
inflation  variable.  Table  5-10  contains  anticipated  inflation  as  well  as  the  actual  inflation 
for  each  year.  OUSD  publishes  new  inflation  rates  in  January  of  each  year  at  which  time 
budgets  are  adjusted  to  account  for  anticipated  inflation.  If  the  actual  inflation  is  higher 
than  anticipated,  then  the  budget  was  not  sufficient  to  cover  the  project  and  an  overrun 
occurred.  The  opposite  would  be  true  for  actual  inflation  being  less  than  anticipated;  the 
budget  would  be  overfunded  resulting  in  a  cost  underrun.  This  research  tries  to  link 
human  capital  to  overruns  so  it  is  important  to  remove  the  effects  of  unexpected  inflation. 
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Model 


The  basic  Cobb-Douglas  production  function  is  given  by:  (Cobb  and  Douglas, 

1928). 

Q  =  AK^U  (2) 

Where  Q  is  a  measure  of  output,  A  is  the  technological  parameter,  K  is  the  capital,  L  is 
the  labor  and  the  exponents  are  some  proportionate  value  summing  to  one  of  K  and  L. 
This  model  indicates  that  as  more  capital  or  labor  is  added  then  more  output  is  achieved. 
This  model  was  adapted  by  adding  another  input  variable,  education,  by  Lynch  &  Black 
(1995). 

Q  =  AKVUE8  (3) 

For  this  research,  Q  is  the  cost  overrun,  K  is  the  capital  of  defense  industries,  L  is  a 
vector  of  the  four  occupational  areas,  E  is  the  education  level  of  those  individuals  and  the 
exponents  again  are  some  percentage  summing  to  one.  We  conducted  a  log-log 
transfonnation  on  equation  (3)  to  linearize  the  specification  for  ease  of  estimation. 

In  Q  =  pinK  +  ylnLFM  +  ClnLEN  +  LlnLPM  +  (4) 

qlnLRated  +  51nE  +aX  +  s 

We  observed  negative  values  in  four  of  the  15  dependent  variable  observations. 
This  indicates  for  those  4  years,  ASC  actually  incurred  an  underrun  or  programs  came  in 
under  cost.  As  such,  we  are  not  able  to  conduct  a  natural  log  transfonnation  on  those 
negative  values.  Therefore,  we  added  the  same  minimal  constant  to  each  of  our 
dependent  variables  to  get  positive  values  for  all  observations.  Now  that  we  have 
positive  values  for  each  year,  we  are  able  to  log  transform  our  dependent  variables.  This 
method  introduces  some  bias  in  the  interpretation  of  our  coefficients  which  we  will 
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address  in  chapter  five.  We  assumed  the  capital  of  defense  industries  is  a  good  proxy  for 
the  capital  variable  K.  Following  the  model  specification  used  by  Feuring  (2007),  the 
yearly  defense  industrial  production  index  published  by  the  Federal  Reserve  Bank  was 
used  because  defense  contractors  utilize  their  own  capital  in  support  of  R&D  contracts. 
The  use  of  industrial  production  data  is  a  standard  proxy  for  capital  in  the  literature 
(Feuring,  2007).  We  added  the  additional  X  variable  to  represent  the  vector  of  control 
variables  discussed  in  the  previous  section.  We  had  zero  and  negative  observations  for 
the  unexpected  inflation  variable  so  we  did  not  conduct  a  natural  log  transformation  on 
unexpected  inflation.  We  will  interpret  the  coefficient  as  a  semi-log  if  found  statistically 
significant. 

We  omitted  the  technological  parameter  A  in  the  original  Cobb-Douglas 
production  function  in  the  final  model  specification.  When  the  final  model  is  log 
transformed,  the  technological  parameter  A  becomes  the  intercept  in  the  linear  equation. 
Since  this  study  is  concerned  with  the  slope  of  the  Cobb-Douglas  function  or  the  effects 
of  human  capital  on  cost  overruns,  the  A  parameter  is  given  a  value  of  one  with  no  loss  of 
generalizability  (Feuring,  2007). 

Summary 

This  chapter  covered  the  model  to  be  used  in  analyzing  the  effects  of  human 
capital  on  cost  overruns  and  which  variables  will  be  used  in  the  analysis.  The  method  of 
collection  for  the  dependent  and  independent  variables  and  why  they  are  important  to  this 
study  was  also  covered.  The  next  chapter  will  present  the  results  of  the  analysis. 
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IV.  Results 


We  start  by  plotting  cost  overruns  over  time  against  the  number  of  engineers, 
program  managers,  rated  personnel,  and  cost  analysts  to  get  a  visual  representation  of  the 
data.  Each  of  the  four  career  fields  are  plotted  on  separate  scatter  plots  to  visualize  the 
effects  of  each  career  field  on  cost  overruns.  Due  to  limitations  of  the  data,  our  study 
ranged  from  1994  through  2008,  or  represents  only  15  years  worth  of  data.  Our  research 
does  not  attempt  to  forecast  future  relationships.  We  are  only  concerned  with  uncovering 


historical  relationships,  if  any,  between  human  capital  and  cost  overruns. 


Figure  7  Scatter  Plots  of  Each  Career  Field  on  Cost  Overruns 

There  appears  to  be  a  relationship  between  the  numbers  of  personnel  in  each  career  field 
and  cost  overruns.  The  scatter  plots  visually  imply  that  decreasing  the  number  of 
engineers,  program  managers,  and  rated  officers  will  cause  the  cost  overruns  to  drift 
toward  0%  and  in  some  cases,  actually  go  negative  indicating  cost  underruns.  The 
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opposite  is  true  for  financial  managers.  If  you  increase  the  number  of  financial  managers, 
the  cost  overruns  begin  to  improve  as  represented  visually  in  Figure  7. 

The  summary  statistics  for  our  database  are  listed  in  Table  1  below.  The  mean 
cost  overrun  for  our  study  is  6.78%  with  a  range  of  -5.89%  to  27.69%.  The  financial 
managers  range  from  385  to  476  with  a  mean  of  437.  The  number  of  program  managers 
range  from  8 12  to  1208  with  a  mean  of  960.  The  number  of  rated  personnel  range  from 
65  to  298  with  a  mean  of  128.  Lastly,  the  number  of  engineers  range  from  1942  to  2603 
with  a  mean  of  2 108.  Our  last  variable  of  interest,  education,  indicates  the  percentage  of 
our  sample  as  having  a  graduate  degree  or  higher  averaged  56.57%  and  ranged  from 
47.69%  to  62.30%.  This  information  is  used  later  in  the  chapter  for  interpretation  of  the 
significant  coefficients. 

Table  1  Summary  Statistics 


Variable 

Mean 

Maximum 

Minimum 

Std.  Dev. 

Observations 

OVERRUNS 

6  78% 

27.69% 

-5  89% 

11.06% 

15 

FINANCIAL  MANAGERS 

437 

476 

385 

30.97 

15 

PROGRAM  MANAGERS 

960 

1208 

812 

106.84 

15 

RATED  PERSONNEL 

128 

298 

65 

66.33 

15 

ENGINEERS 

2108 

2603 

1942 

197.34 

15 

EDUCATION 

56.57% 

62.30% 

47.69% 

4.01% 

15 

CAPITAL 

1062 

119.4 

92.1 

7.51 

15 

RDTE  BUDGET  (CY08SM) 

3553.76 

5151.44 

1980.03 

930.88 

15 

DEFENSE  EMPLOYMENT  (K) 

2828.67 

3850.00 

2180.00 

617.40 

15 

UNEXPECTED  INFLATION 

0.21% 

1.50% 

-0.60% 

0.57% 

15 
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Pre-Estimation  Specification  Tests 

A  stationary  time  series  regression  model  is  one  whose  probability  distributions 
are  stable  over  time.  When  variables  in  a  time  series  regression  are  stationary, 
conventional  statistical  measures  such  as  t-statistics  and  r-squared  are  the  standard 
indicators  used  to  assess  the  performance  of  the  hypothesized  model.  However,  if  the 
variables  are  non-stationary,  such  usual  measures  are  no  longer  valid  in  the  interpretation 
of  the  model  and  can  lead  to  faulty  conclusions  and  spurious  regression.  If  two  non- 
stationary  variables  that  do  not  have  a  relationship  like  the  height  of  a  tree  and  Gross 
Domestic  Product  (GDP)  are  regressed,  it  will  appear  the  height  of  the  tree  is  affecting 
GDP  (Feuring,  2007).  Therefore  our  dependent  and  independent  variables  must  be 
stationary  to  successfully  interpret  our  time  series  regression.  A  common  procedure 
throughout  the  literature  for  ensuring  stationarity  is  to  test  the  variables  using  the 
Augmented  Dickey-Fuller  test.  The  variable  is  replaced  with  the  change  in  that  variable 
from  one  year  to  the  next,  called  the  first  difference,  if  the  variable  is  found  to  be  non- 
stationary.  Using  the  Augmented  Dickey-Fuller  test,  financial  managers,  capital, 
RDT&E  budget,  and  unexpected  inflation  variables  all  required  a  first  difference  to 
become  stationary.  The  program  managers  and  education  variables  required  a  second 
difference  while  the  defense  employment  variable  required  a  third  difference  to  become 
stationary.  The  results  of  our  Augmented  Dickey-Fuller  tests  are  located  in  Appendix  A. 

The  next  pre-estimation  specification  test  was  to  detennine  the  appropriate  lag 
lengths  of  the  independent  variables.  The  theory  behind  lags  is  the  effect  of  an 
independent  variable  may  not  be  felt  until  later  time  periods  in  the  dependent  variable 
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(Feuring,  2007).  Often,  the  lag  length  is  dictated  by  the  frequency  of  the  data  as  well  as 
the  sample  size  and  usually  one  or  two  lags  will  suffice  for  annual  data  (Wooldridge, 
2006).  There  are  several  lag  length  selection  criteria  available.  However,  according  to 
Liew’s  (2004)  research,  the  Akaike  information  criterion  (AIC)  is  recommended  for 
samples  with  60  observations  or  less  so  we  used  the  AIC  to  determine  the  appropriate  lag 
length  of  our  sample  of  15  observations.  To  determine  the  appropriate  lag  length,  we 
regressed  up  to  two  lags  of  each  independent  variable  on  the  dependent  variable  and  the 
results  of  the  AIC  and  adjusted  R  squared  are  shown  in  table  2. 

Table  2  Lag  Structure  Determination 


1st  Difference  Financial 
Managers 

AIC  Adjusted  R* 

2nd  Difference  Program 
Managers 

AIC  Adjusted  R* 

Rated  Personnel 

AIC  Adjusted  R* 

No  Lag 
One  Lag 
Two  Lags 

4.29  0.04 

4.24  0.14 

4.21  0.24 

4.28  0.10 

4.56  -0.09 

4  68  -0.07 

4.38  -0.08 

4.39  -0.06 

4.36  0.11 

Engineers 

AIC  Adjusted  Rz 

2nd  Difference 
Education 

AIC  Adjusted  Rz 

1st  Difference  Capital 
AIC  Adjusted  R'1 

No  Lag 
One  Lag 
Two  Lags 

4.36  -0.06 

4.39  -0.06 

4.38  0.00 

4.47  -0.10 

4.57  -0.10 

4.68  -0.10 

4.41  -0.08 

4.44  -0.06 

4.51  -0.03 

1st  Difference  RDT&E 
AIC  Adjusted  R* 

3rd  Difference  Defense 
Employment 

AIC  Adjusted  R* 

1st  Difference 
Unexpected  Inflation 
AIC  Adjusted  R^ 

No  Lag 
One  Lag 
Two  Lags 

4.28  0.06 

4.43  -0.04 

4.55  -0.07 

4.36  0.03 

4.57  -0.10 

4.57  0.01 

4.40  -0.07 

4  48  0.00 

4.43  0.05 

In  seven  of  the  nine  variables,  the  AIC  was  minimized  when  the  adjusted  R 
squared  was  maximized.  The  RDT&E  and  defense  employment  variables  were  the  other 
two  variables  where  the  minimized  AIC  and  maximized  adjusted  R  squared  were 
different  so  they  were  defaulted  to  a  no  year  lag.  Financial  managers  and  rated  personnel 
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were  maximized  at  lags  of  two  and  all  of  the  other  variables,  either  by  default  or  by 
maximization,  were  at  no  lags.  The  financial  managers  and  rated  personnel  two  year  lag 
is  reasonable  because  both  of  those  career  fields  rotate  in  and  out  of  the  acquisition  field. 
Often,  these  personnel  are  assigned  to  complex  acquisition  programs  so  it  can  take  a  few 
years  for  them  to  become  proficient  in  their  new  acquisition  jobs.  As  such,  it  is  sensible 
to  assume  their  contributions  may  not  be  felt  for  up  to  two  years,  hence  the  two  year  lag. 

Post-Estimation  Specification  Tests 

Diagnostic  checks  were  then  completed  to  ensure  the  model  specification  did  not 
violate  any  regression  assumptions.  Tests  for  multi-collinearity,  heteroskedasticity,  auto¬ 
correlation,  cointegration,  and  nonnality  were  conducted. 

The  Variance  Inflation  Factor  (VIF)  was  calculated  to  test  for  multi-collinearity. 
The  results  of  those  calculations  are  found  in  Appendix  B.  The  VIF  value  should  be 
below  10  to  ensure  no  collinearity  between  the  independent  variables.  The  mean  VIF  for 
our  model  was  6.78  so  our  model  does  not  exhibit  the  presence  of  multi-collinearity. 

We  empirically  tested  for  heteroskedasticity  using  the  Breusch-Pagan-Godfrey 
test.  The  results  of  that  test  yielded  a  p  value  of  .2338  which  indicates  we  cannot  reject 
the  null  hypothesis  of  homoskedasticity. 

We  tested  for  auto  correlation  using  the  Durbin-Watson  test  statistic  which  was 
2.0067.  A  table  of  5%  significance  for  Durbin-Watson  test  statistics  using  15 
observations  and  9  variables  resulted  in  a  lower  bound  of .  175  and  an  upper  bound  of 
3.216  in  order  to  not  be  auto-correlated.  As  such,  our  2.0067  score  is  well  within  that 
range  so  we  do  not  have  auto-correlation. 
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Due  to  the  presence  of  unit  roots  and  lagging  some  of  our  variables,  the  next  post¬ 
estimation  specification  test  was  to  ensure  there  was  no  cointegration  of  the  error  terms. 
For  cointegration,  a  pair  of  integrated,  or  smooth  series,  must  have  the  property  that  a 
linear  combination  of  them  is  stationary  (Granger,  2003).  The  Augmented-Dickey  Fuller 
test  was  conducted  on  the  residuals  to  ensure  they  are  stationary.  The  results  of  the  test 
are  located  at  the  end  of  Appendix  A.  The  result  was  a  test  statistic  of  -3.74  giving  a  p- 
value  of  .0016  indicating  the  residuals  are  stationary. 

The  last  test  was  to  check  for  normality  in  our  model  specification.  We  tested  for 
nonnality  using  the  Jarque-Bera  test  statistic.  The  Jarque-Bera  test  indicated  a  p-value  of 
.8781,  not  enough  to  reject  the  null  hypothesis  of  a  normal  distribution. 


Results 


The  results,  with  the  required  specification  changes  based  upon  the 


aforementioned  diagnostic  tests,  are  displayed  in  table  3  below. 


Table  3  Time  Series  Regression  Results 


Number  of  Observations 
R-Squared 

15 

0.61 

In  (Overruns) 

Coefficients 

Standard 

Errors 

t  values 

Probability 

1st  Difference  In  (Financial  Managers  (Lag  2))” 

-13.602 

4.737 

-2.872 

0.046 

2nd  Difference  In  (Program  Managers) 

-0.230 

2283 

-0.101 

0.926 

In  (Rated  Personnel  (Lag  2))* 

4.570 

2.036 

-2.244 

0.065 

In  (Engineers) 

2.925 

2.537 

1.153 

0.333 

2nd  Difference  In  (Education)* 

-2.586 

1.280 

-2.021 

0.093 

1st  Difference  In  (Capital) 

3.530 

4.248 

0.831 

0.467 

1st  Difference  In  (RDT&E) 

0.989 

0.905 

1.092 

0.375 

3rd  Difference  In  (Defense  Employment) 

-1.768 

2442 

-0.724 

0.522 

1st  Difference  Unexpected  Inflation 

-1.281 

22.673 

-0.056 

0.959 

constant 

-8.791 

6.683 

-1.315 

0.280 

’Significant  to  the  10%  level 
’’Significant  to  the  5%  level 
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The  time  series  regression  results  illustrate  financial  managers,  rated  personnel, 
and  education  are  all  significantly  correlated  with  cost  overruns.  Due  to  the  small  sample 
size  of  our  data,  we  discuss  the  statistical  inferences  in  terms  of  95%  confidence  intervals 
instead  of  point  estimates. 

Specifically,  the  results  show  that  by  adding  more  financial  managers,  cost 
overruns  will  decrease.  For  every  1%  increase  in  financial  managers,  the  cost  overrun 
percentage  may  decrease  anywhere  from  between  4.13%  to  23.08%.  These  results  are 
similar  to  Fuering’s  (2007)  thesis  except  our  research  takes  into  account  more  control 
variables  and  limits  our  scope  of  research  to  only  ASC  managed  programs.  The  average 
number  of  financial  managers  is  437  so  if  ASC  added  1%  more,  or  approximately  4  more 
analysts,  cost  overruns  would  be  expected  to  decrease  by  the  above  mentioned  range. 

Increasing  the  number  of  rated  personnel  is  also  correlated  with  cost  overruns. 

For  every  1%  change  in  the  number  of  rated  personnel,  cost  overruns  would  be  expected 
to  decrease  between  .5%  and  8.64%.  If  rated  personnel  assigned  to  ASC  were  increased 
by  1%,  cost  overruns  on  average  would  decrease  somewhere  between  .034%  and  .59% 
(average  cost  overrun  =  6.78%  *  .5%  or  8.64%).  Both  .034%  and  .59%  seem  like 
insignificant  amounts  but  when  we  considered  the  average  total  RDT&E  budget,  .034% 
and  .59%  equate  to  a  range  of  $1,208  million  to  $20,963  million  (average  RDTE  budget 
=  $3,553  million  *  .034%  or  .59%). 

The  last  statistically  significant  variable,  education,  also  revealed  a  negatively 
correlated  relationship  with  our  dependent  variable.  If  we  increase  the  percentage  of 
graduate  degrees  or  higher  by  1%,  cost  overruns  should  decrease  by  a  range  of 
approximately  .03%  to  5.15%. 
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There  were  no  other  statistically  significant  variables  found  in  our  regression. 
Unfortunately  due  to  the  small  sample  size  of  only  15  years  worth  of  data,  we  are  not  able 
to  make  statistically  significant  inferences  regarding  these  marginally  significant 
variables  but  it  is  interesting  to  note  the  sign  on  the  coefficient  of  those  variables.  For 
example,  the  sign  on  the  coefficients  of  program  managers  and  defense  employment 
indicate  that  increasing  these  variables  may  lead  to  reductions  in  cost  overruns. 
Conversely,  increasing  engineers,  capital  and  RDT&E  may  actually  lead  to  more 
overruns.  More  program  managers  and  individuals  doing  defense  related  work  should 
cause  overruns  to  decrease  because  those  individuals  could  better  support  complex 
acquisition  programs.  However,  the  results  of  increasing  the  number  of  engineers  seem 
counterintuitive.  Maybe  ASC  has  reached  a  point  of  diseconomies  of  scale  with  respect 
to  engineers  and  adding  more  engineers  to  program  offices  actually  would  make  those 
program  offices  less  efficient.  It  is  reasonable  to  assume  increasing  capital  and  RDT&E 
without  corresponding  respective  human  capital  increases  could  possibly  lead  to  poorer 
contract  performance  with  regards  to  cost  overruns. 
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V.  Discussion  of  Results  and  Recommendations 


In  chapter  four,  our  results  yielded  one  independent  variable  statistically 
significant  at  the  .05  level  and  two  independent  variables  statistically  significant  at  the 
.10  level.  Also  in  the  previous  chapter,  we  briefly  discussed  the  signs  on  the  coefficients 
of  the  other  independent  variables  as  well.  Recognizing  we  had  a  very  small  sample  size, 
and  the  fact  that  we  added  a  constant  to  our  dependent  variable  (discussed  in  chapter 
three),  we  felt  it  is  important  to  discuss  the  implications  and  limitations  of  our  findings. 

First,  remember  from  chapter  three  that  we  added  a  minimal  amount  to  our 
dependent  variable  so  we  could  log  transfonn  it  keeping  it  consistent  with  the  Cobb- 
Douglas  production  function.  This  added  constant  introduces  some  bias  to  our 
specification.  Specifically,  we  added  .059  to  each  observation  of  our  dependent  variable. 
Due  to  this  constant  being  a  very  small  amount,  the  resulting  bias  is  almost  non-existent 
because  the  sum  of  the  actual  percentage  and  our  added  constant  are  then  log 
transformed.  However,  this  bias,  coupled  with  our  small  sample  size  obligates  us  to 
discuss  the  findings  in  terms  of  ranges  versus  point  estimates. 

Second,  should  the  Air  Force  now  adopt  policy  changes  based  upon  this  analysis 
alone?  Probably  not,  but  the  results  should  definitely  be  considered.  We  discussed  our 
results  in  terms  of  confidence  intervals  of  the  effects  on  the  cost  overrun  variable.  We 
used  95%  confidence  intervals  in  our  interpretations.  This  is  where  policy  makers  need 
to  reflect  on  the  cost  versus  benefit  ratios  of  hiring  additional  personnel.  Our  results 
show  by  increasing  the  financial  manager,  rated  officer,  and  education  level  variables  the 
cost  overruns  will  decrease  by  some  range.  However  due  to  the  small  sample  size  of  our 
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data,  should  the  policy  makers  consider  those  results  as  being  exact  and  factual?  We 
prefer  to  look  at  in  the  following  manner. 

The  policy  makers  should  believe  the  hiring  of  additional  people  will  equate  to 
less  cost  overruns  based  upon  our  analysis,  albeit  arguable  in  some  cases,  or  they  can 
assume  status  quo  and  hope  the  overruns  will  decrease.  More  research  needs  to  be 
conducted  in  this  area.  Specifically,  we  needed  more  ASC  data  to  have  a  larger  sample 
size  increasing  the  statistical  power  of  the  analysis.  Perhaps,  this  research  could  be 
expanded  to  include  the  other  AFMC  centers  or  the  Air  Force  as  a  whole.  Additionally, 
the  research  could  be  extended  across  the  entire  DoD  to  see  if  similar  results  are  found,  or 
more  importantly,  so  the  sample  size  is  increased  thereby  improving  the  credibility  of  the 
results. 

The  effect  of  program  executive  officers  (PEOs)  and  program  directors  is  an  area 
that  needs  further  study.  Most,  if  not  all,  program  offices  at  ASC  are  managed  by 
personnel  above  the  grade  of  lieutenant  colonel  or  equivalent  and  we  assume  that  logic 
holds  true  across  the  other  AFMC  centers  and  services  as  well.  Our  research  was  limited 
to  personnel  in  the  grades  of  lieutenant  colonel  and  below  or  equivalent.  Unfortunately, 
the  data  for  those  higher  ranked  personnel  was  not  easily  available  so  we  could  not 
attempt  to  capture  the  effects.  We  do  feel  it  is  important  to  mention  that  in  Rusnock’s 
(2008)  thesis,  she  found  the  tenure  of  PEO’s  to  be  statistically  significant  and  negatively 
correlated  on  the  amount  of  cost  growth  in  space  acquisitions.  It  would  be  interesting  to 
see  if  the  same  effect  would  hold  true  for  cost  overruns  or  within  ASC. 

Another  area  that  needs  further  study  is  the  delineation  between  education  and 
training.  We  would  have  preferred  to  model  APDP  levels  versus  graduate  degrees  or 
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higher  as  our  education  variable.  However,  APDP  levels  for  Air  Force  personnel  are  not 
centrally  managed  and  easily  available.  Acknowledging  an  individual  with  a  graduate 
degree  or  higher  will  more  than  likely  have  greater  analytical  skills,  DoD  acquisition  is  a 
unique  place  which  is  why  acquisition  reform  initiatives  require  specialty  training  in  each 
field.  Additionally,  the  personnel  modeled  in  our  research  could  have  held  graduate 
degrees  not  remotely  related  to  their  career  field.  For  example,  a  financial  analyst  may 
have  a  graduate  degree  in  history.  Several  studies  done  on  reform  initiatives  yielded 
mixed  results  but  we  could  not  find  any  research  that  looked  specifically  at  the  effects  of 
APDP  levels  and  cost  overruns.  The  education  variable  could  be  substituted  with  a 
percentage  of  personnel  with  the  appropriate  APDP  level  for  each  career  field,  indicating 
they  had  completed  the  required  prerequisites  for  the  function  they  were  perfonning. 

Further  study  needs  to  be  completed  on  the  effects  of  the  DoD  procurement 
holiday  that  occurred  in  the  early  to  mid  1990s.  The  DoD  as  a  whole  went  through  times 
of  shrinking  budgets  and  personnel.  Unfortunately,  due  to  the  data  available,  our 
research  starts  in  1994  which  is  right  in  the  midst  of  the  aforesaid  holiday.  Further 
studies  need  to  be  accomplished  several  years  out  into  the  future  to  eliminate  the  effects 
of  the  drawdown  in  terms  of  resources,  including  both  dollars  and  people.  Also,  the 
global  war  on  terror  (GWOT)  began  during  the  period  of  our  research.  However,  we  feel 
we  adjusted  for  GWOT  by  only  including  the  RDT&E  budget  in  our  control  variables. 
Most  of  the  increased  spending  that  occurred  during  GWOT  took  place  in  other  fund 
types  like  procurement  and  operating  and  maintenance  accounts.  Unfortunately,  the  DoD 
may  be  on  the  path  to  another  period  of  reduced  spending  (“Fox  News,”  2009)  so  further 
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research  needs  to  be  conducted  on  the  effects  of  the  ebbs  and  flows  of  defense  spending 
on  major  acquisition  programs. 

Last,  more  research  needs  to  be  done  on  the  effects  of  contractors  doing  cost 
analysis,  program  management  and  engineering  duties  in  program  offices.  We  used 
individuals  doing  defense  related  work  in  the  private  sector  as  a  proxy  for  contractors. 
More  detailed  analysis  needs  to  be  done  with  regard  to  contractors  doing  what  used  to  be 
accomplished  by  military  and  government  employees.  ASC  needs  to  better  capture  the 
type  and  amount  of  work  that  has  been  outsourced  to  contractors  to  ascertain  their  effect. 
In  the  future,  someone  could  add  contractor  workers  as  a  variable  to  mathematically 
detennine  if  their  efforts  are  more  or  less  productive  than  military/govemment 
employees. 

In  conclusion,  our  research  identified  a  relationship  between  human 
capital  and  cost  overruns.  In  previous  literature,  links  between  performance  levels  and 
human  capital  have  been  established.  Using  the  lessons  learned  from  previous  studies 
and  the  theoretical  relationship  of  inputs  and  outputs  of  the  Cobb-Douglas  production 
function,  we  presented  analysis  supporting  a  relationship  exists  between  financial 
managers,  rated  personnel,  education,  and  cost  overruns.  By  weighing  the  costs 
associated  with  acquiring  more  personnel  and  funding  more  education  versus  the 
potential  savings  associated  with  fewer  cost  overruns,  ASC  could  potentially  save 
significant  amounts  of  money. 
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Appendix  A  Augmented  Dickey-Fuller  Test  Results 


To  ensure  stationarity  among  our  variables,  EViews6  student  version  was  used  to 
run  the  Augmented  Dickey-Fuller  tests  for  unit  roots.  We  selected  the  trend  and  intercept 
toggle  button  and  chose  the  Akaike  Info  Criterion  (AIC)  for  lag  length  for  each  of  our 
variables.  We  log  transfonned  all  of  our  variables  with  the  exception  of  the  unexpected 
inflation  variable  due  to  the  presence  of  negative  values.  The  results  of  the  E Views 
output  for  each  of  our  variables,  including  residuals,  are  presented  here. 


Unit  Root  Tests 


Null  Hypothesis:  LOG(OVERRUNS)  has  a  unit  root 

Exogenous:  Constant,  Linear  Trend 

Lag  Length:  3  (Automatic  based  on  AIC,  MAXLAG=3) 


t-Statistic  Prob.* 

Augmented  Dickey-Fuller  test  statistic 

-11.31515  0.0001 

Test  critical  values:  1%  level 

-5.124875 

5%  level 

-3.933364 

10%  level 

-3.420030 

Null  Hypothesis:  D(LOG(FINANCIAL_MANAGERS))  has  a  unit  root 

Exogenous:  Constant,  Linear  Trend 

Lag  Length:  0  (Automatic  based  on  AIC,  MAXLAG=3) 


t-Statistic 

Prob. 

Augmented  Dickey-Fuller  test  statistic 

6.662998 

o.ooo: 

Test  critical  values:  1%  level 

4.886426 

5%  level 

3.828975 

10%  level 

3.362984 
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Null  Hypothesis:  D(LOG(PROGRAM_MANAGERS),2)  has  a  unit  root 

Exogenous:  Constant,  Linear  Trend 

Lag  Length:  0  (Automatic  based  on  AIC,  MAXLAG=3) 


t-Statistic 

Prob.* 

Augmented  Dickey-Puller  test  statistic 

-4.080343 

0.0107 

Test  critical  values:  1%  level 

-4.121990 

5%  level 

-3.144920 

10%  level 

-2.713751 

Null  Hypothesis:  LOG(RATED_PERSONNEL)  has  a  unit  root 

Exogenous:  Constant,  Linear  Trend 

Lag  Length:  4  (Automatic  based  on  AIC,  MAXLAG=4) 

t-Statistic  ProbC 

Augmented  Dickey-Puller  test  statistic 

-7.540773  0.0009 

Test  critical  values:  1%  level 

-5.295384 

5%  level 

-4.008157 

10%  level 

-3.460791 

Null  Hypothesis:  LOG(ENGINEERS)  has  a  unit  root 

Exogenous:  Constant,  Linear  Trend 

Lag  Length:  0  (Automatic  based  on  AIC,  MAXLAG=3) 


t-Statistic  Prob.* 

Augmented  Dickey-Puller  test  statistic 

-4.380208  0.0051 

Test  critical  values:  1%  level 

-4.004425 

5%  level 

-3.098896 

10%  level 

-2.690439 
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Null  Hypothesis:  D(LOG(EDUCATION),2)  has  a  unit  root 

Exogenous:  Constant,  Linear  Trend 

Lag  Length:  1  (Automatic  based  on  AIC,  MAXLAG=3) 


t-Statistic  ProbC 

Augmented  Dickey-Puller  test  statistic 

-4.973134  0.0032 

Test  critical  values:  1%  level 

-4.200056 

5%  level 

-3.175352 

10%  level 

-2.728985 

Null  Hypothesis:  D(LOG(CAPITAL))  has  a  unit  root 

Exogenous:  Constant,  Linear  Trend 

Lag  Length:  0  (Automatic  based  on  AIC,  MAXLAG=3) 


t-Statistic  Prob.* 

Augmented  Dickey-Puller  test  statistic 

-5.485077  0.0042 

Test  critical  values:  1%  level 

-4.886426 

5%  level 

-3.828975 

10%  level 

-3.362984 

Null  Hypothesis:  D(LOG(RDTE_BUDGET))  has  a  unit  root 
Exogenous:  Constant,  Linear  Trend 

Lag  Length:  0  (Automatic  based  on  AIC,  MAXLAG=3) 

t-Statistic  Prob.* 

Augmented  Dickey-Puller  test  statistic 

-5.128962  0.0017 

Test  critical  values:  1%  level 

-4.057910 

5%  level 

-3.119910 

10%  level 

-2.701103 
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Null  Hypothesis:  D(LOG(DEFEN SE_EMPLOYMENT),3)  has  a  unit  root 

Exogenous:  Constant,  Linear  Trend 

Lag  Length:  0  (Automatic  based  on  AIC,  MAXLAG=3) 


t-Statistic  Prob.* 

Augmented  Dickey-Luller  test  statistic 

-4.805065  0.0040 

Test  critical  values:  1%  level 

-4.200056 

5%  level 

-3.175352 

10%  level 

-2.728985 

Null  Hypothesis:  D(UNEXPECTED  INFLATION)  has  a  unit  root 

Exogenous:  Constant,  Linear  Trend 

Lag  Length:  1  (Automatic  based  on  AIC,  MAXLAG=3) 


t-Statistic  Prob.* 

Augmented  Dickey-Fuller  test  statistic 

-3.481932  0.0023 

Test  critical  values:  1%  level 

-2.771926 

5%  level 

-1.974028 

10%  level 

-1.602922 

Null  Hypothesis:  RESIDUALS  has  a  unit  root 
Exogenous:  None 

Lag  Length:  0  (Automatic  based  on  AIC,  MAXLAG=2) 


t-Statistic  Prob.* 

Augmented  Dickey-Fuller  test  statistic 

-3.736327  0.0016 

Test  critical  values:  1%  level 

-2.792154 

5%  level 

-1.977738 

10%  level 

-1.602074 
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Appendix  B  Variance  Inflation  Factors 


To  check  for  multi-collinearity  among  our  independent  variables,  we  computed 
the  Variance  Inflation  Factor  (VIF)  for  each  of  our  independent  variables.  We  did  this  by 
running  an  ordinary  least  squares  regression  that  has  each  independent  variable  as  a 
function  of  all  the  other  explanatory  variables.  We  took  the  resulting  r-squared  values 
and  calculated  the  VIF  for  each  variable  using  the  following  equation. 

VIF(fit)  =  ^  (5) 

Next  we  averaged  all  of  the  VIFs  to  arrive  at  the  mean  VIF  for  the  variables  in  the  model. 


Variable 

VIF 

Financial  Managers 

3.56 

Program  Managers 

4.35 

Rated  Personnel 

10.99 

Engineers 

10.63 

Education 

4.09 

Capital 

6.71 

RDT&E  Budget 

12.90 

Defense  Employment 

4.75 

Unexpected  Inflation 

3.08 

Mean  VIF 

6.78 
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